Repository navigation
Add vg lift: lift features to the graph, the gref and every path - #5110
Open
glennhickey wants to merge 1 commit into
Open
glennhickey wants to merge 1 commit into
glennhickey wants to merge 1 commit into
Conversation
A tool that scans a cover (the gref paths of a GBZ, or the tigs vg unitigs writes) reports what it finds in the coordinates of the sequence it read, and those coordinates mean nothing to anything else. vg lift translates them back: it takes a BED on a cover sequence or on any path in the GBZ and writes a GAF whose records carry each feature's graph walk, how many non-gref paths visit the whole walk (sp:i:), its gref placement (gr:Z:), its path coordinate (pc:Z:) and its BED name (an:Z:). A feature that several sequences spell at the same graph place is reported once. The path coordinate names a place in the graph without depending on node IDs: PATH:START-END:STRAND on the first path that carries the whole feature, taking the gref path it lies along before a reference or assembly path. Paths keep their coordinates across clipped, filtered and sampled graphs, and no two places in a graph share a coordinate. With a VCF, each ALT allele becomes a feature: the allele's walk without the first and last nodes, which sit on the reference on either side. The walk comes from INFO/AT, or from a PanGenie variant ID for a single-ALT record without one. The output VCF carries each allele's gref placement in INFO/GR and its path coordinate in INFO/PC. Where an allele runs along a gref alt contig and differs from it -- an insertion carrying SNPs, say -- each difference becomes a biallelic record on that contig, written after the calls the way vg deconstruct writes nested records. A haplotype gets 1 where its called allele has the variant, 0 where the allele reads that stretch of the contig without it, and . otherwise, so frequencies are among the carriers of the contig; GQ is the lowest GQ of the calls behind the genotype. This is what lets a caller's output be read in gref coordinates, where the alt contigs hold the sequence the reference lacks. With --depth it writes how many distinct haplotypes read each base of the gref paths as bedGraph, including runs of depth 0, optionally within regions. Whether a VCF belongs to this graph is checked by looking for each ALT allele in the sequence of its walk, allowing for the padding base at either end. A one-base ALT against a longer REF is only that padding base, since the trimming some panels do takes the allele's own sequence out of the columns, and nothing is then left to test. A GT that names an ALT allele its record does not have is read as the reference and counted, since reading it that way is a guess. Features are read as a VCF only when the path names a regular file that htslib reads as one. Probing a FIFO would read its first block into htslib's detection buffer and throw it away on close, leaving whoever opens the path next in the middle of the file, and a VCF has to be a regular file here in any case, since the lift reads it twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #5109 (
vg unitigs). Review that one first; this diff is against its branch.A tool that scans a cover (the gref paths of a GBZ, or the tigs
vg unitigswrites) reports what it finds in the coordinates of the sequence it read, and those coordinates mean nothing to anything else.vg lifttranslates them back. It takes a BED on a cover sequence or on any path in the GBZ and writes a GAF whose records carry each feature's graph walk and four tags:sp:i:(how many non-gref paths visit the whole walk),gr:Z:(gref placement),pc:Z:(path coordinate) andan:Z:(BED name). A feature that several sequences spell at the same graph place is reported once.The path coordinate names a place in the graph without depending on node IDs:
PATH:START-END:STRANDon the first path that carries the whole feature, taking the gref path it lies along before a reference or assembly path. Paths keep their coordinates across clipped, filtered and sampled graphs, and no two places in a graph share a coordinate.With a VCF, each ALT allele becomes a feature: the allele's walk without the first and last nodes, which sit on the reference on either side. The walk comes from
INFO/AT, or from a PanGenie variant ID for a single-ALT record without one. The output VCF carries each allele's gref placement inINFO/GRand its path coordinate inINFO/PC.Where an allele runs along a gref alt contig and differs from it, an insertion carrying SNPs for instance, each difference becomes a biallelic record on that contig, written after the calls the way
vg deconstructwrites nested records. A haplotype gets 1 where its called allele has the variant, 0 where the allele reads that stretch of the contig without it, and.otherwise, so frequencies are among the carriers of the contig. GQ is the lowest GQ of the calls behind the genotype. This is what lets a caller's output be read in gref coordinates, where the alt contigs hold the sequence the reference lacks.With
--depthit writes how many distinct haplotypes read each base of the gref paths as bedGraph, including runs of depth 0, optionally within regions.Two limits worth knowing before running this on whole genomes:
Tested on macOS:
make -j8clean,scripts/lint.pyclean, andprove t/59_vg_lift.t t/58_vg_unitigs.t t/11_vg_paths.tgreen (228 assertions).59_vg_lift.tpins exact records for nested SNPs in a called insertion, allele counts among contig carriers, a walk read backwards giving the same records, GQ taken from the calls, phased and unphased genotypes, a sample without GQ, contig-end cases, and--depthrows including off-gref bases.🤖 Generated with Claude Code